Progress report versus final report

Although the terms ‘interim report’ or ‘progress report’ may suggest that these are less important than the final report, in fact the opposite is true. The progress report supplies early information and advice, with which the recipients (such as client, project manager and others) can often make timely adjustments for keeping the total process on the right track. The final report is more a retrospective evaluation that mainly benefits subsequent test processes and projects.

Risk Report

The test manager creates a risk report if events take place for which measures are required to be taken that the test manager is not authorised to decide upon. Another reason for creating a risk report is if the client asks the test manager to set out consequences and possible measures for one or more scenarios upon which a decision is required to be taken. For example, a scenario in which the client sees that the development activity is overrunning and he considers making budget available from the test.

Empirical data

Examples of empirical data are:

• Size of the test object
• Development effort
• Number of defects
• Duration and hours per main activity
• Duration and hours required for specifying tests
• Duration and hours required to execute the tests
• Number of test cases
• Analysis of lead time per defect
• Number of defects to be expected
• Number of retests.

A comprehensive summary of the empirical data that can be collected is included in the Metrics list (see Metrics). This guideline also discusses the Goal-Question-Metric method for implementing metrics.

Costs/benefits analysis

The costs of the test process are relatively simple to establish. Bear in mind, for example, the costs of the used resources, manpower and equipment. The benefits of the test process, however, are more difficult to establish. It is difficult, but not impossible, to provide a quantitative indication of these.